On the equivalence of two expected average reward criteria for zero-sum semi-Markov games

In this paper we study two basic optimality criteria used in the theory of zero-sum semi-Markov games. According to the first one, the average reward for player 1 is the lim sup of the expected total rewards over a finite number of jumps divided by the expected cumulative time of these jumps. According to the second definition, the average reward (for player 1) is the lim sup of the expected total rewards over the finite deterministic horizon divided by the length of the horizon. We shall call them the ratio-average reward and time-average reward, respectively. It is known that in general these two criteria can have nothing to do with each other. In other words, they may lead to different rewards and optimal strategies for players. The ratio-average reward is somewhat easier to study and has been used by many authors in zero-sum games and in dynamic programming. Recently, some results concerning the optimality equation for semi-Markov games with Borel state space and the ratio-average criterion were given [1]. However, an equivalence result has not been reported so far for the Borel (uncountable) state space models. The aim of this paper is to show the equivalence between two expected average rewards under some geometric ergodic conditions. At the same time, we prove that the optimality equations for the models with these criteria are the same. Our proof is based on [2] and employs basic facts from renewal theory. Certain consequences of V -geometric ergodicity given in enable us to apply the optional sampling theorem of Doob, which is the core of the proof.